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Introduction 


New teacher evaluation systems are part of a recent wave of education 
reform that has focused on measuring and improving teacher effective- 
ness. Within the last five years, over 40 states have implemented or 
are implementing new teacher evaluation systems that include multiple 
measures such as classroom observations and student growth. 


Chicago is part of that wave. It launched its revised 
teacher evaluation system. Recognizing Educators 
Advancing Chicago Students (REACH) in the fall of 
2012. The REACH system differs significantly from 
the old checklist system: Teachers are observed on 
multiple occasions instead of just once; their observa- 
tion ratings are based on a detailed rubric and evalua- 
tors have to provide evidence for each rating assigned. 1 
Final evaluation scores are based on a combination of 
these observations and student growth metrics which 
can be a combination of student growth on district-wide 
grade-and subject-specific performance tasks for all 
teachers and value-added scores for teachers in grades 
3-8 (For more details on REACH, see Appendix A). 

Many studies have focused on the development or 
technical aspects of the metrics used to capture teacher 
quality, but notably fewer studies have focused on how 
teachers and administrators perceive the new systems. 
To fill this gap, and as part of an implementation study 
of REACH, the University of Chicago Consortium on 
School Research (UChicago Consortium) has been 
studying how Chicago Public School (CPS) teachers and 
administrators perceive this system since its imple- 
mentation in 2012. Prior reports have shown that teach- 
ers and administrators were generally positive about 


Data Sources 


Data in this brief are taken from an annual survey 
of all teachers and principals: 

2014-15 My Voice, My School (MVMS) Survey 

• Administered January-March 2015 

• Survey administered to 24,661 teachers 

• 19,908 responded (81%) 

2014-15 Consortium Administrator Survey 

• Administered May 2015 

• 288 of 643 principals responded (45%) 

• 291 of 589 assistant principals responded (49%) 

For more details on these surveys, and response 
rates for participants who were asked REACH- 
related questions, see Appendix B. 


the new system, especially the observation process, 
but that teachers were negative about the inclusion of 
student growth, whether it was measured by standard- 
ized tests or teacher-created district-wide performance 
tasks. Teachers also felt the new system created ad- 
ditional stress. We noted a slight decline in teacher sat- 
isfaction with the overall system between 2012-13 and 
2013-14 (Years 1 and 2), although teachers’ perceptions 


1 In CPS, only principals and assistant principals can be evaluators. 
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of their evaluator’s fairness and ability to rate them 
remained strong. We also found teacher perceptions 
of REACH were positively related to their perceptions 
of school leadership and school community. 2 

REACH implementation was staggered depending 
on teachers’ tenure status and previous rating. Figure 1 
provides a timeline of REACH implementation and 
survey administration dates; questions on the surveys 
were used to gather teacher and administrator atti- 
tudes about REACH. Those attitudes might have been 
influenced by participants’ level of experience with 
the system at the time they responded to the survey. 

For example, at the time participants responded to the 
first survey, teachers had no knowledge of their student 
growth scores, and limited knowledge of observation 
ratings; administrators had experienced the demands 
of observing non-tenured teachers for stakes, but had 
not yet needed to officially evaluate tenured teachers. 

This brief builds on the prior two reports of teacher 
and administrator perceptions by summarizing their 
responses collected near the end of REACH’S third year. 
It also provides overall district-wide summative scores. 


While it is important to study the technical properties 
of measuring teacher performance, it is also important 
to note that participants have a key role in identifying 
areas of strength and weakness. Examining their per- 
ceptions can provide important pieces of information 
as practitioners and policymakers look to build on the 
strengths of the system now in place and to minimize 
or correct areas of weakness. 

Findings 

In the first year of REACH, we found most teachers 
and principals responded positively to the observa- 
tion process, but teachers were apprehensive about the 
inclusion of student growth in their evaluation. In the 
second year, our findings were similar to the first year: 
teachers continued to be positive about the observation 
process, negative about the inclusion of student growth. 
We also found a decline in teacher satisfaction with 
the overall system in Year 2 compared to the first year. 
It is important to look at participant perceptions in 
Year 3 since their level of experience with the system 
has changed. 


FIGURE 1 

REACH Reporting and Study Data Collection 

Year 1: 2012-13 Year 2: 2013-14 Year 3: 2014-15 Year 4: 2015-16 


Spring 2013 

M VMS 


Teacher Survey 
and UChicago 
Consortium 
Administrator 
Survey 


Fall 2013 

Non-tenured 
teachers receive 
a REACH rating 
from Year 1. 


Spring 2014 

MVMS 


Teacher Survey 
and UChicago 
Consortium 
Administrator 
Survey 


Fall 2014 

Non-tenured 
teachers receive 
a REACH rating 
from Year 2. 


Spring 2015 

MVMS 


Teacher Survey 
and UChicago 
Consortium 
Administrator 
Survey 


Fall 2015 

Tenured and 
non-tenured teachers 
receive a REACH 
rating from Year 3. 


All teachers receive 
REACH report. 


All teachers receive 
REACH report. 


All teachers receive 
REACH report. 


Note: Tenured teachers missing a prior rating or with prior ratings of Unsatisfactory or Satisfactory are observed four times and receive a summative REACH rating 
annually beginning in Year 2. Tenured teachers with prior ratings in the top two categories are observed four times over the course of two years beginning in Year 2 
and receive a summative REACH rating biennially. In Year 1 (2012-13) only non-tenured teachers’ REACH ratings counted for stakes. Non-tenured teachers are observed 
four times and receive a summative REACH rating annually. All teachers received REACH reports annually which detail scores on each measure of REACH. For more 
details on REACH, see Appendix A. 


2 For previous reports, see http://consortium.uchicago.edu/ 
page/teacher-evaluation 
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REACH Final Ratings 


Figure A displays teachers’ ratings for both tenured 
and non-tenured teachers under the prior checklist 
system and under REACH. We see that under REACH 
fewer teachers are in the top two categories. 

Most tenured teachers receive their final REACH 
rating biennially; hence, most tenured teachers received 
their first final rating in 2014-15. In that year, 83 percent of 
tenured teachers received ratings in the top two catego- 
ries. In comparison, under the previous checklist system, 
94 percent of tenured teachers received ratings in the 
top two categories (Superior or Excellent). The shift in 
the percent of teachers in the top category was more 
pronounced, going from 69 percent to 29 percent. 

Under the previous checklist system, 87 percent 
of non-tenured teachers received ratings in the top 
two categories. Under REACH, 58 percent of teachers 
received ratings in top two categories in 2012-13. In 
Year 2 and Year 3 of REACH, we see a shift upward 
in ratings with more non-tenured teachings assigned 
top ratings. At this point, it is unclear whether this 
shift upward is due to teacher attrition, teacher 
improvement, or evaluators becoming more lenient. 


A teacher’s REACH final rating is comprised of a 
professional practice score and up to two measures 
of student growth. 

• Professional Practice is based on four observa- 
tions using the CPS Framework for Teaching. 

• Student Growth can be a combination of the following: 

Value-Added Measures: Teachers teaching read- 
ing and math in grades 3-8 receive an individual 
value-added score. Teachers teaching other 
subjects and grades may receive a school-level 
value-added score in literacy. 

Performance Tasks: Developed by teams of CPS 
teachers, individual schools, and/or central office 
staff, performance tasks are written or hands-on 
assessments designed to measure the mastery or 
progress toward mastery of a particular skill or 
standard. Performance tasks are typically admin- 
istered and scored by teachers. There is a grade/ 
subject specific performance task for every grade 
and subject combination. 

For more details on REACH components, see Appendix A. 


FIGURE A 

Fewer Teachers in Top Two Categories Under REACH 


REACH Final Ratings: Tenured Teachers 


REACH Final Ratings: Non-Tenured Teachers 
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Checklist 
2003-04 to 


REACH 
Year 1 


2007-08 2012-13 


REACH 
Year 2 
2013-14 


REACH 
Year 3 
2014-15 


I Superior 
I Excellent 
Satisfactory 
Unsatisfactory 


I Excellent 
I Proficient 
I Developing 
| Unsatisfactory 


I Superior 
I Excellent 
Satisfactory 
Unsatisfactory 


I Excellent 
I Proficient 
I Developing 
| Unsatisfactory 


Note: Most tenured teachers have received final REACH ratings one time 
(2014-15) as 2012-13 was a pilot year and tenured teachers are rated biennially. 
Figure reflects only tenured teachers who received final ratings in 2014-15. 
Checklist ratings are from 2003-04 to 2007-08. 

Source: Checklist ratings; http://tntp.org/assets/documents/TNTPAnalysis- 
Chicago.pdf 


Note: REACH figures reflect only non-tenured teachers who received final 
ratings in each of the years represented. Checklist ratings are from 2003-04 
to 2007-08. 

Source: Checklist ratings; http://tntp.org/assets/documents/TNTPAnalysis- 
Chicago.pdf 
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Teacher and Administrator REACH Year 3 
Survey Results 3 

The majority of teachers and administrators remain 
optimistic that REACH will improve instruction and 
student learning. In Year 3 of REACH, over 60 percent 
of teachers agreed or strongly agreed they were satis- 
fied with the evaluation system. And over 60 percent of 
teachers and over 80 percent of administrators agreed 
or strongly agreed REACH will lead to better instruc- 
tion and improved student learning (see Figure 2). 

Most teachers continue to have negative perceptions 
about the inclusion of student growth metrics in their 
evaluation. Almost 60 percent of teachers surveyed 
disagreed or strongly disagreed with the statement 
that performance tasks were a fair way to evaluate 
their performance and 72 percent disagreed or strongly 
disagreed that value-added measures were a fair way to 
evaluate their performance (see Figure 3). In addition, 
most teachers continue to feel student growth counts 


as too much of their final score. Over 60 percent of 
teachers reported their evaluation relies too heavily 
on student growth (see Figure 7). 

Most teachers believe their evaluators are fair and 
classroom observations are a fair way to evaluate their 
performance. Seventy-six percent of teachers surveyed 
agreed or strongly agreed using classroom observations 
were a fair way to evaluate their performance (see Figure 3). 
Most teachers continue to report positively when asked 
about their own evaluators. About 70 percent of teachers 
responded ‘to a great extent’ when asked if their evaluator 
was fair, and unbiased, and if their evaluator was able to 
accurately assess their performance. And 63 percent of 
teachers reported their evaluators were knowledgeable 
about their strengths, weaknesses, and classrooms to a 
great extent (see Figure 4). 

Teachers and administrators continue to report 
the observation process has encouraged reflection 
and influenced practice. Almost ninety percent of 


FIGURE 2 

Teachers/Administrators Remain Positive the Evaluation Process Flas Potential to Improve Instruction, 
Student Learning 


To what extent do you Agree or Disagree with the following statements? 
2014-15 MVMS Teacher Survey 


Overall, I am satisfied with the teacher 
evaluation process at this school. 
(n=13,806) 


T 


12% 

26% 

52% 

10% 


The evaluation process will lead to 
improved student learning. (n=13,836) 


48% 


13% 


The evaluation process will lead to 
better instruction at my school. 
(n=13,878) 


9% 2 



51% 


13% 


H Strongly Disagree H Disagree H Agree Strongly Agree 


2014-15 UChicago Consortium Administrator Survey 


The evaluation process will lead to 
improved student learning. (n=525) 


The evaluation process will lead to 
better instruction in my school. (n=525) 



I Strongly Disagree H Disagree H Agree Strongly Agree 


3 In this brief, we do not explicitly make year-to-year comparisons each year. Reports from our early survey findings can be found 

for all survey items as not all survey items were consistent in at http://consortium.uchicago.edu/page/teacher-evaluation. 
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FIGURE 3 


Most Teachers Agree Classroom Observations Are a Fair Way to Evaluate Performance, and Disagree that 
Student Growth Is a Fair Way to Evaluate Performance 

To what extent do you Agree or Disagree with the following statements? 


2014-15 MVMS Teacher Survey 



I Strongly Disagree Disagree Agree Strongly Agree 


Note: Number of respondents on items may be significantly different due to randomization of some survey items. See Appendix B for more details. Perceptions may 
not add to 100 percent due to rounding. 


FIGURE 4 

Most Teachers Continue to Perceive Their Evaluators as Fair, Unbiased, and Knowledgeable about 
Their Classrooms and Practice 

To what extent do you Agree or Disagree with the following statements? 


2014-15 MVMS Teacher Survey 




My evaluator knows what is going 

on in my classroom. 

rn-R 

% 

9% 

28% 59% 






My evaluator is fair and unbiased. 

(n=5,504) 


7o 6% 

20% 71% 




My evaluator knows my strengths 
and weaknesses as a teacher. 

(n=5,511) 



3% 8% 

26% 63% 



My evaluator is able to accurately 

assess my instruction. 2% 

(n=5,506) 



6% 

24% 67% 




M Not at All I A Little I Some To a Great Extent 


Note: The number of respondents on items may be significantly different due to randomization of these survey items. See Appendix B for more details. Percentages may 
not add to 100 percent due to rounding. 
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teachers agreed or strongly agreed the observation 
process has encouraged them to reflect on their prac- 
tice, 88 percent reported it has changed their teaching, 
and 92 percent said they have used the feedback they 
have received (see Figure 5). Administrators also 
report teachers using their feedback and making im- 
provements. Seventy-seven percent of administrators 
report most or all of their teachers have incorporated 
the feedback into their teaching and 61 percent of 
administrators report most or all of their teachers 


have made noticeable improvements over the year 

(see Figure 6). 

Both teachers and administrators continue to 
report the evaluation process has increased teachers’ 
levels of stress and anxiety. And most administrators 
agreed or strongly agreed REACH has increased their 
own levels of stress. In addition, over half of teachers 
and over 40 percent of administrators agreed or 
strongly agreed the evaluation process takes more 
effort than the results are worth (see Figure 7). 


FIGURE 5 

Almost 90 Percent of Teachers Agree or Strongly Agree the Observation Process Has Encouraged 
Reflection and Improved Their Practice 

To what extent do you Agree or Disagree with the following statements? 


2014-15 MVMS Teacher Survey 



■ Strongly Disagree I Disagree ■ Agree Strongly Agree 


Note: The number of respondents on items may be significantly different due to randomization of these survey items. See Appendix B for more details. Percentages may 
not add to 100 percent due to rounding. 


FIGURE 6 

Administrators Report Most Teachers Have Incorporated Evaluation Feedback Into Their Teaching 

Of the teachers you have observed this year, how many... 


2014-15 UChicago Consortium Survey 



I Few or None MAboutHalf Most All 


Note: See Appendix B for more details on the UChicago Consortium administrator survey including response rates. 
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FIGURE 7 

Both Teachers and Administrators Report REACH Has Increased Their Stress and Anxiety 
To what extent do you Agree or Disagree with the following statements? 


2014-15 MVMS Teacher Survey 



H Strongly Disagree H Disagree Agree Strongly Agree 


2014-15 UChicago Consortium Administrator Survey 


The evaluation process has 
increased my level of stress 
and anxiety. (n=522) 


The evaluation process has 
increased teachers’ level of 
stress and anxiety. (n=530) 


The evaluation process takes 
more effort than the results 
are worth. (n=522) 


5% 27% 


44% 


24% 



51% 


30% 


27% 


15% 


I Strongly Disagree I Disagree Agree Strongly Agree 


Note: Number of respondents on items may be significantly different due to randomization of some survey items. See Appendix B for more details on the UChicago 
Consortium administrator survey including response rates. Perceptions may not add to 100 percent due to rounding. 
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Summary 


Teacher and administrator perceptions of REACH 
provide important information about buy-in and about 
the implementation of the initiative. They are based on 
participant experiences with each piece of the system; 
they also depend on their sense of whether it is fair and 
useful and whether it places an undue burden on them. 
Our survey findings from Year 3 of REACH are very 
similar to our findings from Year 2, indicating there 
was little change in perceptions about REACH’S fair- 
ness and usefulness as participants gained more direct 
experience with the system. 

There are some clear positive patterns in responses 
to questions about fairness, potential, and usefulness. 
Three years into implementation, most teachers and 
administrators view many aspects of REACH favor- 
ably and believe it is leading to instructional improve- 
ment. Over all three years, teachers have consistently 
responded that they see their evaluators as fair and able 
to assess their performance. Teachers also responded 
positively when asked whether observations are a fair 
way to assess their performance. Both administrators 
and teachers agreed the feedback from observations are 
being used to improve practice and that the system has 
potential to improve instruction and student learning. 

Similarly, teachers and principals remain negative 
about aspects of REACH that were viewed negatively in 
prior years. Teachers continue to believe that student 
growth counts too much in their evaluation. Slightly 
fewer than four out of every five teachers agreed REACH 


has increased their levels of stress and anxiety and 
more than two out of three of administrators agreed 
that REACH had increased their own levels of stress. 

Both positive and negative perceptions provide 
useful information for the success of this initiative, 
since it relies so heavily on the engagement and partici- 
pation of teachers and school leaders. Previous research 
has shown that the degree to which any educational 
reform is successful at meeting its goals depends on how 
individuals involved in that effort interpret and act on 
that reform. Hence, it is crucial for policymakers and 
practitioners to take participant perceptions about this 
reform into account and build on the positive percep- 
tions while simultaneously addressing those aspects 
participants perceive as needing improvement. 

This is the third report in this series on teacher 
and administrator perceptions of REACH. As new 
evaluation systems are implemented in districts and 
states across the nation, it is an important time to 
understand teacher and administrator response to 
and interpretation of evaluation policies, especially as 
the policy arena around teacher evaluation continues 
to change with the passage of the Every Student a 
Success Act (ESSA). Our future work on teacher 
evaluation in Chicago will continue to highlight teacher 
and administrator perceptions, focusing on how the 
data generated by teacher evaluation systems are used 
and how their use may translate into improvements in 
teacher practice. 
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Appendix A 

2014-15 REACH Scores and Ratings 


A teacher’s REACH score is comprised of a professional 
practice score and up to two measures of student growth. 
For more details on REACH, visit http://www.cps.edu/ 
reachstudents. 

Professional Practice 

Teachers are evaluated over multiple classroom observa- 
tions using the CPS Framework for Teaching, a modi- 
fied version of the Charlotte Danielson Framework for 
Teaching. Formal observations last at least 45 minutes 
and include pre- and post-observation conferences. 
Currently in CPS, only principals and assistant princi- 
pals can be certified evaluators. To be assigned a REACH 
rating, a teacher must be observed four times. Non- 
tenured teachers 4 and tenured teachers with previous 
low ratings are observed four times annually and receive 
a REACH rating each year. Tenured teachers with previ- 
ous high ratings 5 are observed four times over the course 
of two years and receive a REACH rating every two years, 
since under Illinois law tenured teachers are evaluated 
every two years. 

Student Growth 

To meet Illinois state law requirements about which as- 
sessments must be used for teacher evaluation, CPS has 
identified two different types of student assessments. 

Value-Added Measures 

Teachers who teach grades 3-8 reading and/or math 
receive an individual value-added score based on their 
students’ NWEA MAP— an adaptive, computer-based 


test. Teachers in non-tested subjects and/or grades 
receive a school-level literacy value-added score. For 
high school teachers in core subjects, CPS used the 
EPAS suite of tests (EXPLORE, PLAN, and ACT) in 
the 2013-14 school year; no value-added scores were 
used for high school teachers in 2014-15. 

Performance Tasks 

Developed by teams of CPS teachers, individual schools, 
and/or central office staff, performance tasks are writ- 
ten or hands-on assessments designed to measure the 
mastery or progress toward mastery of a particular skill 
or standard. Performance tasks are typically adminis- 
tered and scored by teachers at the beginning and end of 
each year. 

REACH Scores and Ratings 

Professional practice scores are combined with student 
growth scores for an overall REACH score, which rang- 
es from 100 to 400 and translates to a REACH rating 
of Unsatisfactory, Developing, Proficient, or Excellent 
(see Table A.1). The percentages of professional prac- 
tice and student growth are detailed in Table A. 2. 

TABLE A.1 
REACH Ratings 


100 - 209 

Unsatisfactory 

210 - 284 

Developing 

285 - 339 

Proficient 

340 - 400 

Excellent 


REACH Score 


Rating 


4 Most teachers in CPS hired after July 1, 2013 attain tenure at 
the beginning of their fourth or fifth year, depending on their 
evaluation ratings. 

5 Tenured teachers with previous low ratings include those 
who received an Unsatisfactory or Satisfactory rating on the 


previous system. Tenured teachers missing previous ratings 
were to receive four observations and a REACH rating in 2013- 
14 and then be placed on a biennial cycle in the following year. 
Tenured teachers with previous high ratings include those who 
received an Excellent or Superior rating on the previous system. 


UCHICAGO Consortium Research Brief | Teacher Evaluation in Practice: Year 3 


9 




TABLE A. 2 

REACH Measures and Percentages 
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Appendix B 

Survey Data 


DATA 

Data from this brief include surveys of teachers and prin- 
cipals in CPS. Teachers and principals in charter schools 
were not asked REACH-related items. For more informa- 
tion on the survey, including questions that were asked, 
see consortium.uchicago.edu/surveys/documentation. 

CPS’ My Voice, My School Teacher 
(MVMS) Survey 

This web-based survey was conducted by UChicago 
Impact in collaboration with CPS and the Illinois State 
Board of Education. It was administered in January 
through March 2015 to all teachers in all CPS neighbor- 
hood, charter, selective enrollment, and alternative 
schools. 24,661 classroom teachers were eligible to par- 
ticipate; 19,908 responded (response rate=81%). Survey 
questions on the teacher survey included questions on 
leadership, school climate, and teacher collaboration. 
Responses on the MVMS teacher survey are anony- 
mous, so we cannot link, for example, a teacher’s per- 
ceptions about REACH to his or her evaluation ratings. 

Only teachers in non-charter schools were asked 
REACH-related questions since charter schools in CPS do 
not participate in REACH. Of a possible 21,257 teachers 


in non-charter schools, 16,950 (80%) replied to REACH- 
related questions, although the number of respondents 
varied with each question. In an effort to reduce the length 
of teacher time spent on the survey, some REACH-related 
survey questions were randomized; that is, some teachers 
were randomly selected to answer one group of REACH 
items and other teachers were randomly selected to 
answer a different group of REACH items. We tested for 
differences between the randomly assigned groups and 
found no significant differences. 

UChicago Consortium Administrator Survey 

We included REACH-related content on UChicago 
Consortium’s administrator survey. This web-based 
survey was administered to all principals and assistant 
principals in May 2015. Administrators in 532 non- 
charter schools and 111 charter schools were invited 
to participate; a total of 288 principals in charter and 
non-charter schools responded, for an overall principal 
response rate of 45 percent. However, only administra- 
tors in non-charter schools were invited to respond to 
REACH-related questions. 252 of these 532 principals 
responded (47%); 291 assistant principals of a possible 
589 responded (49%). 
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OUR MISSION The University of Chicago Consortium on School Research 
(UChicago Consortium) conducts research of high technical quality that 
can inform and assess policy and practice in the Chicago Public Schools. 
We seek to expand communication among researchers, policymakers, and 
practitioners as we support the search for solutions to the problems of 
school reform. The UChicago Consortium encourages the use of research in 
policy action and improvement of practice, but does not argue for particular 
policies or programs. Rather, we help to build capacity for school reform 
by identifying what matters for student success and school improvement, 
creating critical indicators to chart progress, and conducting theory-driven 
evaluation to identify how programs and policies are working. 



THE UNIVERSITY OF | | | URBAN 

Chicago UEIsr 


ISBN 978- 


0-9975073-1-7 
5 0 9 9 5 > 


9 780997 507317 




